New Structures for Transformation-Based Generators

The Problem: A serious problem of most pattern-directed, transformation-based program generators is that they are trying to achieve three mutually antagonistic goals simultaneously: 1) deeply factored and highly abstract operators and operands to gain the combinatorial programming leverage provided by compositions of abstractions, 2) high performance code in the generated program, and 3) small (i.e., practical) generation search spaces. Various program generators focus on one or two of these goals thereby often compromising the other goal(s). This paper will make the argument that this quandary is due in large measure to control and storage structure deficiencies of conventional transformation-based generators. While pattern-directed (PD) transformations (also called rules) make the specification of the transformations easy and, in fact, mostly do a pretty good job of refining (i.e., translating) domain specific abstractions into code, they often explode the search space when one is later trying to produce highly optimized code from deeply factored, inter-constrained abstract operators and operands. Since giving up the deep factoring of abstractions to reduce the search spaces also gives up the combinatorial programming leverage provided by the composition, it is not a good trade-off. The general form of the constraint propagation problem in program generation is NP complete [3]. What can one do? Don’t solve the general problem! Instead, solve specialized sub-problems that avoid search space explosions via metaprograms that use specialized storage and control structures as well as specialized strategies. Some such strategies and mechanisms have been employed in earlier generators (e.g., Draco) and some are introduced by the Anticipatory Optimization Generator (AOG) generator (e.g., Localization and Tag-Directed rules) [1-2]. Table 1 summarizes these sub-problems, the strategies used and the mechanisms that they employ. The remainder of the paper examines these strategies and their implementation structures. Phased Translation: Draco [4] attacks the problem of the large search spaces by defining distinct Domain Specific Languages (DSLs) and performing program generation by phased translation of higher level DSLs into successively lower level DSLs until a conventional programming language is reached. Since the operators and operands of the DSLs are distinct, this has the effect of implicitly grouping the translation rules so that only those rules relevant to the current DSL construct being translated are tried. In effect, this reduces one very large search space to a series of much smaller search spaces. Domain Specific Optimization: However, this strategy introduces another problem – complex generated code that explodes the number of cases needed to recognize each conceptual construct (e.g., (x + 0) and (0 + x) and x are all really the same). To solve this problem, Draco introduces an optimization phase between each DSL to DSL translation phase. This optimization phase converts the generated DSL code into a simpler, canonical form using rules that map from a domain (i.e., a DSL) to itself. These optimization rules use domain knowledge to perform optimizations. These optimizations are relatively simple using domain knowledge but are often extremely hard to do in a programming language because key domain knowledge has been translated away. Domain knowledge is the key leverage that makes the optimizations easy. Localization: AOG adopts these same strategies as a starting point but adds new strategies. DSLs significantly improve program productivity because they deal with large-grain data structures and large-grain operators and thereby allow a programmer to say a lot (i.e., express a lengthy computation) with a few symbols. Large-grain operators applied to large-grain data structures imply some kind of extended control structure such as a loop, a sequence of statements, a recursive function, or other structure. As one composes large-grain operators and operands together into longer expressions, each subexpression implies not only some computation (e.g., pixel addition) that will eventually be expressed in terms of atomic operators and data items (e.g., integer addition), but it also implies